Emotional Expression in Oral History Narratives: Comparing Results of Automated Verbal and Nonverbal Analyses

Audiovisual collections of narratives about war-traumas are rich in descriptions of personal and emotional experiences which can be expressed through verbal and nonverbal means. We complement a commonly used verbal analysis with a nonverbal one to study emotional developments in narratives. Using automatic text, vocal, and facial expression analysis we found that verbal emotional expressions do not correspond much to nonverbal ones. This observation may have important implications for the way narratives traditionally are being studied. We aim to understand how different modes of narrative expression relate to each other, and to enrich digital audiovisual interview collections with emotion-oriented tags. 1998 ACM Subject Classification I.2.7 Natural Language Processing, H.3.1 Content Analysis and Indexing, H.3.7 Digital Libraries